1. Field of the Invention
The present invention relates to a video system for automatically controlling a play speed based on complexity of video image contents. Particularly, the present invention relates to an intelligent fast-forward digital video system, which indexes entire video contents based on the content complexity and then controls relative play speed of the video based on the indexing information.
More specifically, the present invention relates to a method of defining the video content complexity using the information reflecting substantially the meanings of video content development, a method of controlling play speed based on the defined complexity, a method of selectively reproducing frames to play MPEG video at the FF mode without adjusting a frame rate, and a system for realizing various playing speeds in accordance with the moving picture species or contents or user's taste and demand.
2. Background of the Related Art
In a conventional speed-shift mode of moving pictures, the shift speed is fixed to a constant value through all the sections on speed shifting.
However, such a speed shifting technology fails to satisfy the visual characteristics of a user when the regeneration is played at a Fast or Slow mode.
For instance, when the regeneration is played at a fast uniform speed, the user is unable to recognize the scene due to the fast transformation of the abrupt change of pictures according to the time. On the other hand, when the change of pictures is slow, the regeneration is played too slow to realize the fast display.
Moreover, when the regeneration is played slow, the user hardly recognize the wanted pictures passing too fast during the abruptly-varying scenes as well as feels tedious for the slow scenes running slower.
If the FF mode is executed for high-speed search, the slow-forward mode is carried out for looking into specific scenes in detail. Considering the substantial user's visual aspect, the user expects to slow-shift of the scenes during the abrupt-changed scenes, and vice versa.
Unfortunately, the fixed fast/slow shift play fails to meet the user's demand.
Therefore, the above problem may be solved by automatically controlling the play speed of moving pictures by adaptively changing the play speed in accordance with the picture change ratio in a speed shift interval.
Such a technology of an automatic controlling method of adaptive speed of moving pictures is disclosed in Korean Laid-open No. 1000-00813 published in Nov. 15, 1999, in which the speed in the shift regenerative interval is adapted to the human visual characteristics in accordance with the moving picture characteristics by varying the play speed in the executed intervals fast or slow in accordance with the movement change ratio of the pictures by detecting cuts and using the picture difference between the frame number among the cuts and the adjacent frame.
FIG. 1 shows a block diagram of a system for automatically controlling an adaptive speed of moving pictures.
Referring to FIG. 1, a speed control information detection module 102 detects the moving picture variation for controlling the play shift speed in a shift interval of a digital moving picture file 101. And, the speed control information detection module 102 further consists of a cut detection module 102a extracting cuts from moving pictures and calculating the number of frames between the cuts and a differential picture detection module 102b finding and storing a differential picture between the frames in the moving pictures. A first storage module 103 stores moving pictures data for speed shift which contain the detection information of the cut detection module 102a and the differential picture detection module 102b. A second storage module 104 stores moving picture data for constant speed. And, a display module 105 controls the display speed instantly fast or slow in a shift interval by judging the variation degree of the moving pictures from the frame number between the cuts and the differential pictures between the frames.
The cut detection module 102a extracts and stores the cut detection for judging the scene conversion, indexing of the detected cuts, and information of a distance(frame number) between the cuts.
The differential picture detection module 102b stores the variation degree of scenes by detecting the differential pictures between the respective frames.
The display module 105 has an algorithm related to the speed and the distance between the cuts enabling to control the speed through the distance(frame number) between the extracted cuts when an FF mode is selected and another algorithm related to the speed and the detected scene variation ratio and speed when a slow-forward(SF) mode is selected.
A conception of a system for controlling play speeds automatically in accordance with the image complexity has been introduced by a display system for controlling a play speed automatically in accordance with the variation ratio of scenes. Yet, this conception defines the scene variation as a differential picture by ‘image complexity’ based on an moving picture difference between adjacent frames.
Therefore, in case of defining the motion as the differential picture, long processing time is required because entire pixels should be compared.
Moreover, when the image complexity is defined by the differential picture, in the case of an having great differences in background and color despite having the same motions, the motion of the object is over-calculated despite a small motion so as to be regenerated slowly. Thus, the motion of the object fails to reach a wanted play speed substantially. Namely, in order to understand the development of a video story substantially, the image complexity is attained by the differential moving pictures by regarding the complexity of the image itself as a content complexity instead of adjusting the play speed by putting a much value on a meaningful story development.
Besides, the difference between the foregoing and following pictures in the substantial story development of a video is unable to consider the meanings of the story sufficiently. And, the complexity of the story contents may be understood differently in accordance with the story development as well as the simple difference of the pictures. For instance, it is assumed that the conversation scene of the characters in a drama goes on tediously, which may be judged as a complicated image due to the picture difference caused by the frequently-changed locations of a camera. Therefore the regeneration of the scene is played slowly. Yet, it would be better to play this scene part fast even though the conversation scene of the characters goes on tediously somewhat long.
The above problem is caused by the difficulty in judging with the differential pictures by calculating automatically ‘content complexity’ which reflects the substantial meaning of contents in accordance with the genre, use, and story of moving pictures. In an aspect of real content development of a video, the ‘content complexity’ of a video reflects the ‘content complexity’ felt by a user only when motion information, shot information, face information(detection information of a specific object as a crucial element of the content development) of characters, text information, audio information and the like as well as the differential pictures are totally taken into consideration.